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This study uses a measure based on 
grade 9 course credits earned and fail- 
ures to examine four-year high school 
graduation rates for students classified as 
"on track" and "off track" to graduate in 
two urban Midwest Region districts. For 
both districts, the on-track indicator was 
a significant predictor of on-time high 
school graduation, even after controlling 
for student background characteristics 
and grade 8 assessment test scores. 

Recent estimates suggest that of U.S. public 
high school freshmen in the fall of 2005 24.5 
percent did not graduate on time in 2008/09 
(Stillwell, Sable, and Plotts 2011). 1 As states 
and school districts attempt to boost gradua- 
tion rates, they face the challenge of identify- 
ing which students are at risk of not graduat- 
ing on time. Early warning indicators based on 
measurable student outcomes and behaviors 
could help identify students at risk while there 
is still time to redirect their trajectory away 
from dropping out or falling behind. 

The current study focuses on the freshman on- 
track indicator developed by the Consortium on 
Chicago School Research (CCSR). This indicator 
identifies a student at the end of the first year of 
high school as on track to graduate if: 

• The student has accumulated the neces- 
sary course credits in grade 9 to move to 
grade 10, as defined by district policy. 



• The student has no more than one semes- 
ter failure in any English, math, science, or 
social studies course. 

The CCSR found that of the Chicago fresh- 
man class of 1999, 81 percent of students who 
were on track graduated from high school 
within four years compared with 22 percent of 
students who were off track (Allensworth and 
Easton 2005). 

Educators want to know how consistently in- 
dicators predict graduation across districts. In 
a recent study, Regional Educational Labora- 
tory (REL) Southwest found that in five Texas 
districts, a greater percentage of on-track 
than off-track students graduated within four 
years (Hartman et al. 201 1). 2 However, across 
Chicago and the five Texas districts studied by 
Hartman et ah, the difference in on-time grad- 
uation rates for on-track and off-track students 
varied considerably, ranging from 18.4 per- 
centage points to 59.0 percentage points. This 
report adds to this body of research by apply- 
ing the CCSR model to two additional districts 
(referred to as District A and District B), both 
in the Midwest Region. 

Although the CCSR study of Chicago students 
found that the on-track indicator had predic- 
tive value after accounting for student back- 
ground characteristics (Allensworth and Easton 
2005), that study cannot predict whether this 
is the case in other districts. The current study 
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SUMMARY 



examines the extent to which the CCSR on-track 
indicator predicts on-time graduation in two 
other districts after controlling for student back- 
ground characteristics and prior achievement. 

The current study examined three research 
questions for two urban districts in the Mid- 
west Region: 

• What were the freshman on-track and off- 
track rates for recent cohorts, overall and 
by student background subgroup? 

• How did four-year in-district graduation 
rates compare for on-track and off-track 
freshmen in recent cohorts, overall and by 
student background subgroup? 

• To what extent does the on-track indica- 
tor predict four-year graduation rates for 
recent cohorts in each district, after ac- 
counting for baseline student background 
characteristics? 

The main results of the study are the following: 

• For both districts, students who were on 
track at the end of grade 9 graduated on 
time at a higher rate than did students 
who were off track. This was the case both 
overall and for every student background 
subgroup examined in each district. 

• In District A, the graduation rate was 
80.7 percent for on-track students and 
30.2 percent for off-track students in 
the 2005/06 cohort and 77.7 percent 
and 30.0 percent in the 2006/07 cohort. 

• In District B, the graduation rate was 
90.6 percent for on-track students and 



46.1 percent for off-track students in the 
2005/06 cohort and 90.5 percent and 
44.7 percent in the 2006/07 cohort. 3 

• For both districts, the on-track indicator 
was a significant predictor of on-time high 
school graduation, even after controlling 
for student background characteristics 
and for student assessment test scores in 
grade 8. The odds of on-time graduation 
for students who were on track at the end 
of their freshman year was estimated to be 
6.6 times that of students who had similar 
characteristics but were off track at the 
end of their freshman year for District A 
and 5.5 times for District B. 

• For both districts, the effect size of being 
on track compared with being off track 
(as measured by increases in the odds of 
graduating) was larger than the effect size 
for every student background characteris- 
tic and for grade 8 assessment test scores. 

• In District A, the percentage of students 
who were on track at the end of grade 9 
ranged from 41.1 percent to 51.5 per- 
cent across four cohorts. The on-track 
and off-track rates varied with sub- 
group classifications based on gender, 
race/ethnicity, individualized education 
program (IEP) status, age, and grade 8 
proficiency level on the state math and 
reading assessments. The on-track rate 
for individual subgroups ranged from 
25.3 percent to 73.9 percent. 

• In District B, the percentage of stu- 
dents who were on track at the end 
of grade 9 ranged from 84.6 percent 
to 86.8 percent across five cohorts. 
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Notes 



The on-track and off-track rate varied 
with subgroup classifications based 
on gender, race/ethnicity, IEP status, 
free or reduced-price lunch status, age, 
and grade 8 proficiency level on the 
state math and reading assessments. 
The on-track rates for individual 
subgroups ranged from 67.2 percent to 
97.6 percent. 

The pattern of results in this study is similar to 
that of prior studies (Allensworth and Easton 
2005; Hartman et al. 2011), with some differ- 
ences in overall on-track rates, on-track rates 
for particular student subgroups, and the 
degree to which the on-track indicator differ- 
entiates between graduates and nongraduates. 



1. Stillwell, Sable, and Plotts (201 1) applied the av- 
eraged freshman graduation rate, a method used 
by the National Center for Education Statistics 



to estimate the percentage of public high school 
freshmen who graduate with a regular diploma 
four years after starting grade 9 (Seastrom et al. 
2006a, 2006b). Under this method, the rate for 
2008/09 (the freshman class in the fall of 2005) 
equals the total number of diploma recipients 
in 2008/09 divided by the average enrollment of 
grade 8 students in 2004/05, grade 9 students in 
2005/06, and grade 10 students in 2006/07. 

2. Both Allensworth and Easton (2005) and Hart- 
man et al. (2011) calculated graduation rates by 
tallying the number of on-track and off-track 
freshmen in each district who went on to gradu- 
ate on time. Students who left the district during 
grade 9 or who transferred into the district after 
grade 9 were not included in these calculations. 
The same method was used in this report. 

3. The findings for the first two research questions 
show that on-track rates and on-time gradu- 
ation rates were higher for District B than for 
District A. It is not within the scope of this 
study to compare the two districts, however, or 
to explain the underlying reasons for the on- 
track rates or graduation rates in either district. 

April 2012 
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This study uses 
a measure based 
on grade 9 course 
credits earned and 
failures to examine 
four-year high school 
graduation rates for 
students classified as 
"on track" and "off 
track" to graduate in 
two urban Midwest 
Region districts. For 
both districts, the 
on-track indicator 
was a significant 
predictor of on- 
time high school 
graduation, even 
after controlling for 
student background 
characteristics and 
grade 8 assessment 
test scores. 



WHY THIS STUDY? 

Recent estimates suggest that 24.5 percent of U.S. 
public high school freshmen in the fall of 2005 did 
not graduate on time in 2008/09 (Stillwell, Sable, 
and Plotts 2011). 1 As states and school districts 
attempt to boost graduation rates, they face the 
challenge of identifying which students are at risk 
of not graduating on time. Early warning indica- 
tors based on measurable student outcomes and 
behaviors could help identify students at risk while 
there is still time to change their trajectory away 
from dropping out or falling behind. 

Multiple studies have shown that racial/ethnic 
minority students and economically disadvan- 
taged students are less likely to graduate on time 
than are their White and more affluent counter- 
parts (Aud et al. 2010; Beyond High School 2011; 
Orfield et al. 2004; Stillwell et al. 2011; Swanson 
2009). According to one analysis, nationwide 
graduation rates in 2008/09 were 65.9 percent 
for Hispanic students and 63.9 percent for Black 
students, compared with 82.0 percent for White 
students (Stillwell et al. 2011). According to school 
administrators, in 2007/08 the average rate of 
graduation with a diploma for grade 12 students 
in high-poverty secondary schools was 68 percent, 
an 18 percentage-point drop from 1999/2000. By 
contrast, 86 percent of grade 12 White students in 
low-poverty secondary schools graduated with a 
diploma in 2007/08, unchanged from the rate in 
1999/2000 (Aud et al. 2010). 2 

The consequences of not graduating from high 
school are well documented for students and so- 
ciety (for example, Kaplan, Damphousse, and Ka- 
plan 1994; U.S. Bureau of the Census 2006; Rouse 
2007; Muennig 2007; Cutler and Lleras-Muney 
2008; Levin 2009). Nongraduates have lower life- 
time earnings (Muennig 2007; U.S. Bureau of the 
Census 2006), poorer health (Cutler and Lleras- 
Muney 2006; Kaplan et al. 1994; Rouse 2007), 
and shorter life spans (Wong et al. 2002). High 
school dropouts are more likely to be unemployed, 
to receive public assistance (Muennig 2007), to 
commit crimes (Levin 2009), to have children at a 
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USING THE FRESHMAN ON-TRACK INDICATOR TO PREDICT GRADUATION IN TWO URBAN DISTRICTS 



Early, accurate 
identification of at-risk 
students may support 
the development of 
effective, efficient 
intervention programs to 
boost graduation rates 



younger age, and to be single par- 
ents (Kaufman, Alt, and Chapman 
2004). Because of these conse- 
quences, states and school districts 
in the United States are under 
pressure to improve high school 
graduation rates, particularly for 
racial/ethnic minority students 
and students from disadvantaged 
backgrounds (Balfanz et al. 2007; Klemick 2007). 



Early, accurate identification of at-risk students 
may support the development of effective, efficient 
intervention programs to boost graduation rates 
(Hauser and Koenig 2011; Heppen and Therriault 
2008; Jerald 2006, 2007). As states make strides 
in assembling longitudinal data systems, they 
are interested in developing indicators based on 
data from their student information systems that 
can be used to identify students who are at risk. 3 
Studies of middle school students in Philadelphia 
(Neild and Balfanz 2006; Neild and Farley 2004) 
and high school students in Chicago (Allensworth 
and Easton 2005, 2007) have shown that student 
outcomes and behaviors measured in middle 
school and early high school, including poor 
academic performance, low attendance, and lack 
of progression from one grade to the next, are 
correlated with the likelihood of not graduating on 
time. These academic and behavioral factors were 
also found to more accurately predict graduation 
than were student background characteristics (Al- 
lensworth and Easton 2007; Jerald 2006; Neild and 
Balfanz 2006). These findings suggest that states 
and school districts could incorporate indica- 
tors associated with dropping out into their data 
systems so that at-risk students could be identified 
while there is still time to intervene (Hauser and 
Koenig 2011). 



The Consortium on Chicago School 
Research freshman on-track indicator 

The current study focuses on an early warning 
indicator developed by the Consortium on Chicago 
School Research (CCSR), which takes into ac- 
count grade 9 course credits and failures in core 



courses. This freshman on-track indicator, based 
on research in the Chicago Public Schools district, 
considers a student to be on track at the end of the 
first year of high school if: 

• The student has accumulated the necessary 
course credits in grade 9 to move to grade 10, 
as defined by district policy. 

• The student has no more than one semester 
failure in any English, math, science, or social 
studies course. 

A student who fails to meet either of these criteria 
is considered off track at the end of the first year of 
high school. 

CCSR researchers found a relationship between 
graduation and course credits earned and number 
of course failures in freshman year (Allensworth 
and Easton 2005, 2007). The number of absences 
and grade point average were also found to be 
associated with graduation. The on-track indica- 
tor was based on credit accumulation and course 
failures because these are two key components re- 
quired for graduation. A student who falls behind 
in freshman year credits will need to accelerate 
credit accumulation in subsequent years, increas- 
ing the difficulty of graduating on time. Likewise, 
a student who fails a course required for gradu- 
ation will have to take the course again, possibly 
delaying graduation. 

The CCSR found that the on-track indicator pre- 
dicted graduation among Chicago Public Schools 
students. For students entering high school in 
1999/2000, 81 percent of on-track students and 
22 percent of off-track students graduated from 
high school within four years (Allensworth and 
Easton 2005). For students entering high school in 
2000/2001, 82 percent of on-track students and 22 
percent of off-track students graduated from high 
school within four years. A recent Regional Educa- 
tional Laboratory (REL) Southwest study applied 
the CCSR indicator to five school districts in Texas 
and in all cases found a higher on-time gradua- 
tion rate for students identified as on track than 
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for students identified as off track (Hartman et al. 
2011). All these studies found that the on-track 
indicator predicted on-time graduation not only 
overall, but also for several student background 
subgroups, including racial/ethnic minority and 
economically disadvantaged students, who tend to 
have lower graduation rates. 

These findings allow two possible interpretations. 
One is that the on-track indicator differentiates 
graduates from nongraduates because it reflects 
aspects of student background and prior academic 
experience that are themselves associated with 
graduating on time. For example, students who are 
low-achieving or from disadvantaged backgrounds 
might be more likely to be off track in their fresh- 
man year and to not graduate on time. Alterna- 
tively, the on-track indicator might predict student 
graduation over and above any association it has 
with student background and prior academic 
experience. That is, freshman credits and course 
failures may be related to graduation even after 
student background has been taken into account. 



graduating on time (within four years). 4 The cur- 
rent study provides information for two of these 
districts (referred to here as District A and District 
B), each with 30,000-40,000 students, on how well 
the on-track indicator differentiates students who 
graduated on time from those who did not. 5 

For the two freshman cohorts discussed in the 
CCSR reports (Allensworth and Easton 2005, 2007), 
the difference in on-time graduation rates between 
Chicago students identified as on track and those 
identified as off track in grade 9 was 59 percent- 
age points for one cohort and 60 percentage points 
for the other. In the Texas school districts (Hart- 
man et al. 2011), the difference ranged from 18.4 
percentage points to 51.7 percentage points. The 
current study, by analyzing two more districts, will 
give educators access to an expanded repository of 
district-level studies to consider as they design local 
early warning systems. Educators can benefit from 
additional information about how consistently the 
CCSR on-track indicator predicts on-time gradua- 
tion across districts with different characteristics. 



To see how the two interpretations applied to 
Chicago students, CCSR researchers examined the 
relationship between on-track status and on-time 
graduation while controlling for student gender, 
race/ethnicity, economic status, age, and prior 
assessment test scores. (Allensworth and Easton 
2005, 2007). A relationship between the on-track 
indicator and on-time graduation remained even 
after student background characteristics were 
accounted for. The on-track indicator also pre- 
dicted graduation for subgroups of students based 
on gender, race/ethnicity, economic background, 
and prior achievement. The current study uses a 
similar analysis to examine how the on-track in- 
dicator relates to on-time graduation, controlling 
for student background characteristics and prior 
assessment test scores. 



Study motivation and research questions 

Three Midwestern urban districts asked REL 
Midwest to explore the CCSR on-track indicator as 
one potential tool to identify students at risk of not 



The current study also examines how well the 
CCSR on-track indicator predicts on-time gradu- 
ation when controlling for student background 
characteristics and prior achievement. The ben- 
efit of this analysis is that it provides additional 
information about how well the on-track indicator 
identifies students who do and do not graduate on 
time, regardless of certain student characteristics. 
Although CCSR found that the on-track indicator 
predicted graduation for Chicago students after 
accounting for student background variables, it is 
unknown whether this is the case in other districts. 



The study addresses 
the following research 
questions: 

• What were the fresh- 
man on-track and 
off-track rates for re- 
cent cohorts, overall 
and by student back- 
ground subgroup? 



The current study 
examines how 
well the on-track 
indicator predicts 
on-time graduation 
when controlling for 
student background 
characteristics and 
prior achievement 
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• How did four-year in-district graduation rates 
compare for on-track and off-track freshmen 
in recent cohorts, overall and by student back- 
ground subgroup? 

• To what extent does the on-track indicator 
predict four-year graduation rates for recent 
cohorts in each district, after accounting for 
baseline student background characteristics? 

The study uses the CCSR method for calculating 
freshman on-track and off-track rates for the total 
student population and for specific student back- 
ground subgroups. For the 2005/06 and 2006/07 
cohorts, the study compares the on-time gradua- 
tion rates for students identified as on track or off 
track at the end of their first year of high school. 
For these two cohorts, the relationship between 
on-track status and on-time graduation is also 



analyzed, using a regression model that controls 
for student background variables that could be as- 
sociated with graduating on time. The study data 
and methods are described further in box 1 and in 
appendix A. 



This section presents the findings by research ques- 
tion for two urban districts in the Midwest Region. 



Freshman on-track and off-track graduation rates 

District A. The share of freshmen who were on- 
track to graduate in the four cohorts in District A 
was highest in the 2005/06 cohort (51.5 percent) 
and decreased for each subsequent cohort, reach- 
ing 41.1 percent for the 2008/09 cohort (figure 1). 



STUDY FINDINGS 



BOX 1 

Study data and analysis 

Data sources. Both districts provided 
student-level data for the 2005/06, 
2006/07, 2007/08, and 2008/09 fresh- 
men cohorts. District B also provided 
data for the 2009/10 freshman cohort. 
(These data were not available for 
District A.) The data files included 
the following information for each 
student: district entrance date and 
(if applicable) exit date and exit code, 
course records (including grades and 
credits earned), birth date, gender, 
race/ethnicity, individualized educa- 
tion program status, limited English 
proficiency status, free or reduced- 
price lunch status for District B (these 
data were not available for District A), 
and grade 8 scores and proficiency 
levels on state math and reading 
assessments. 

Definitions of on-track and off-track 
status. Students were identified as 
on track at the end of their first 



year of high school if they had ac- 
cumulated the credits needed to 
be promoted to grade 10 as defined 
by district policy and if they had 
received no more than one failing 
grade in a semester in a core subject 
(English, math, science, or social 
studies). Students were identified as 
off track at the end of the first year 
of high school if they failed to meet 
either of these criteria. 

Definition of on-time graduation 
status. Students in the 2005/06 and 
2006/07 cohorts were classified as 
having graduated on time if dis- 
trict files identified them as having 
graduated on a date that was within 
four years of the date they began 
high school (see appendix B for more 
detail). 

Analysis. For each cohort in each dis- 
trict included under the first research 
question on freshman on-track and 
off-track rates, the percentage of stu- 
dents who were on track and off track 



at the end of their first year of high 
school was calculated overall and for 
subgroups (gender, race/ethnicity, 
individualized education program 
status, age at the start of grade 9, and 
proficiency level on grade 8 math and 
reading assessments). 

For the two cohorts in each district 
under the second research question 
on four-year in-district graduation 
rates, the percentage of on-track and 
off-track students who graduated 
within four years of starting high 
school was calculated overall and for 
the student subgroups. 

For the two cohorts in each district 
under the third research question 
on how well the on-track indicator 
predicts four-year graduation rates, 
the relationship between the on- 
track indicator and the probability of 
student graduation was examined, 
controlling for student background 
characteristics and for grade 8 math 
and reading assessment scores. 



STUDY FINDINGS 



5 



FIGURE 1 

Percentage of freshmen in the 2005/06-2008/09 
cohorts in District A identified as on track or off 
track at the end of grade 9 

Percent ■ On track ■ Offtrack 




2005/06 2006/07 2007/08 2008/09 

(n = 2,941) (n = 2,810) (n = 2,666) (n = 2,771) 

Note: On-track and off-track rates are for first-time freshmen enrolled in 
grade 9 at the end of the 2005/06-2009/10 school years for whom on- or 
off-track status could be determined. Because of incomplete course 
records, on-track status could not be determined for some students in 
district A: 95 students in the 2005/06 cohort, 68 in the 2006/07 cohort, 

95 in the 2007/08 cohort, and 89 in the 2008/09 cohort. 

Source: Authors' analysis of student records provided by District A. 

Figure 2 presents freshman on-track and off-track 
rates by student background subgroup. Differences 
in on-track rates were observed across classifica- 
tions within each subgroup. For each cohort, 
on-track rates were higher for female students, 
White students, students without an individual- 
ized education program (IEP), students who were 
the traditional age of a high school freshman (age 
14 or under at the start of freshman year), and 
students who met or exceeded proficiency on the 
state math or reading assessment in grade 8. As 
in the overall sample, the on-track rate for most 
subgroups was highest in the 2005/06 cohort and 
decreased in each subsequent cohort. 

District B. The on-track rate in the five cohorts 
in District B ranged from 84.6 percent for the 
2006/07 cohort to 86.9 percent for the 2009/10 
cohort (figure 3). The rate did not consistently 
increase or decrease across cohorts. 

Figure 4 presents freshman on-track and off- 
track rates by student background subgroup. 



Differences in on-track rates were observed across 
classifications within each subgroup. For each 
cohort, on-track rates were higher for female stu- 
dents, White students, students without an IEP, 
students who were not eligible for free or reduced- 
price lunch, students who were the traditional 
age of a high school freshman (age 14 or under 
at the start of freshman year), and students who 
met or exceeded proficiency on the state math or 
reading assessment in grade 8. As in the overall 
sample, the on-track rate for most subgroups did 
not consistently increase or decrease across the 
cohorts. An exception was students who did not 
meet proficiency on the grade 8 state reading 
assessment; their on-track rate was highest in the 
2005/06 cohort and decreased in each subsequent 
cohort. 



Four-year in-district graduation rates for 
on-track and off-track freshmen 

For both districts, the four-year graduation rate 
was calculated for on-track and off-track students 
in the 2005/06 and 2006/07 cohorts. 

District A In both cohorts, the four-year gradua- 
tion rate for on-track students (79.8 percent in the 
2005/06 cohort and 76.9 percent in the 2006/07 
cohort) was approximately 2.5 times that for off- 
track students (29.9 percent in both the 2005/06 
cohort and the 2006/07 cohort; figure 5). 6 

Within all subgroups, the on-time graduation rate 
was higher among students who were on track to 
graduate in four years than among students who 
were off track (figure 6). (See table D1 in appendix 
D for overall graduation rates for on-track and off- 
track students combined.) 

Gender. For female students, the on-time gradu- 
ation rate was 81.6 percent for on-track students 
and 34.1 percent for off-track students in the 
2005/06 cohort and 78.4 percent and 34.8 percent 
in the 2006/07 cohort. For male students, the on- 
time graduation rate was 77.5 percent for on-track 
students and 26.4 percent for off-track students 
in the 2005/06 cohort and 74.8 percent and 25.9 
percent in the 2006/07 cohort. 
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USING THE FRESHMAN ON-TRACK INDICATOR TO PREDICT GRADUATION IN TWO URBAN DISTRICTS 



FIGURE 2 

Percentage of freshmen in the 2005/06-2008/09 cohorts in District A identified as on track or off track at the 
end of grade 9, by student background subgroup 

Percent A. Gender ■ On track ■ Offtrack 
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2007/08 
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Wofe.- On-track and off-track rates are for first-time freshmen enrolled in grade 9 at the end of the 2005/06-2009/10 school years for whom on- or off-track 
status could be determined. Data were not available to create subgroups based on free or reduced-price lunch status. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of non- 
Black minority students. 

b. Students with missing assessment scores were not included in the calculations for math and reading proficiency levels. The state classifies assessment 
scores into five proficiency categories. The proficient category and two above-proficient categories were combined to create the "met or exceeded profi- 
ciency" category; two below-proficient categories were combined to create the "did not meet proficiency" category. 

Source: Authors' analysis of student records provided by District A. 
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FIGURE 3 

Percentage of freshmen in the 2005/06-2009/10 
cohorts in District B identified as on track or off 
track at the end of grade 9 

Percent ■ On track ■ Offtrack 




2005/06 2006/07 2007/08 2008/09 2009/10 

(n = 2,777) (n = 2,798) (n = 2,853) (n = 3,029) (n = 2,969) 

Note: On-track and off-track rates are for first-time freshmen enrolled in 
grade 9 at the end of the 2005/06-2009/10 school years for whom on- or 
off-track status could be determined. Because of incomplete course re- 
cords, on-track status could not be determined for some students in District 
B: 106 students in the 2005/06 cohort, 102 in the 2006/07 cohort, 105 in the 
2007/08 cohort, 79 in the 2008/09 cohort, and 76 in the 2009/10 cohort. 

Source: Authors' analysis of student records provided by District B. 

Race/ethnicity. For White students, the on-time 
graduation rate was 86.0 percent for on-track stu- 
dents and 29.3 percent for off-track students in the 
2005/06 cohort and 82.1 percent and 28.7 percent in 
the 2006/07 cohort. For students in the other race/ 
ethnicity subgroup, the on-time graduation rate was 
77.4 percent for on-track students and 30.0 percent 
for off-track students in the 2005/06 cohort and 74.6 
percent and 30.1 percent in the 2006/07 cohort. 

Individualized education program status. For stu- 
dents with an IEP, the on-time graduation rate was 
70.9 percent for on-track students and 28.2 percent 
for off-track students in the 2005/06 cohort and 61.7 
percent and 28.4 percent in the 2006/07 cohort. For 
students without an IEP, the on-time graduation rate 
was 82.1 percent for on-track students and 30.5 per- 
cent for off-track students in the 2005/06 cohort and 
79.7 percent and 30.4 percent in the 2006/07 cohort. 

Age. For grade 9 students who were the typical age 
of a high school freshman (age 14 or under at the 



start of grade 9), the on-time graduation rate was 

83.6 percent for on-track students and 37.2 percent 
for off-track students in the 2005/06 cohort and 

81.7 percent and 37.7 percent in the 2006/07 co- 
hort. For students who were older than the typical 
high school freshman (over age 14 at the start of 
grade 9), the on-time graduation rate was 68.9 
percent for on-track students and 22.2 percent for 
off-track students in the 2005/06 cohort and 63.4 
percent and 19.9 percent in the 2006/07 cohort. 

Grade 8 math proficiency. For students who met 
or exceeded proficiency on the grade 8 math as- 
sessment, the on-time graduation rate was 90.2 
percent for on-track students and 49.8 percent for 
off-track students in the 2005/06 cohort and 85.9 
percent and 39.2 percent in the 2006/07 cohort. 

For students who did not meet proficiency on the 
grade 8 math assessment, the on-time graduation 
rate was 72.4 percent for on-track students and 
27.3 percent for off-track students in the 2005/06 
cohort and 62.6 percent and 24.7 percent in the 
2006/07 cohort. 

Grade 8 reading proficiency. For students who met 
or exceeded proficiency on the grade 8 reading 
assessment, the on-time graduation rate was 86.3 
percent for on-track students and 39.4 percent for 
off-track students in the 2005/06 cohort and 84.3 
percent and 40.1 percent in the 2006/07 cohort. 

For students who did not meet proficiency on the 
grade 8 reading assessment, the on-time gradu- 
ation rate was 69.3 percent for on-track students 
and 23.8 percent for off-track students in the 
2005/06 cohort and 61.4 percent and 21.6 percent 
in the 2006/07 cohort. 

District B. In both cohorts, the four-year gradua- 
tion rate for on-track students (90.6 percent in the 
2005/06 cohort and 90.5 percent in the 2006/07 
cohort) was nearly twice the rate for off-track stu- 
dents (46.1 percent in the 2005/06 cohort and 44.7 
percent in the 2006/07 cohort; figure 7). 7 ’ 8 

Within all student background subgroups, on- 
time graduation rates were higher among students 
who were on track to graduate in four years than 
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USING THE FRESHMAN ON-TRACK INDICATOR TO PREDICT GRADUATION IN TWO URBAN DISTRICTS 



FIGURE 4 

Percentage of freshmen in the 2005/06-2009/10 cohorts in District B identified as on track or off track at the 
end of grade 9, by student background subgroup 
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Note: On-track and off-track rates are for first-time freshmen enrolled in grade 9 at the end of the 2005/06-2009/10 school years for whom on- or off-track status 
could be determined. 



a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of non-Black 
minority students. 

b. Students with missing assessment scores were not included in the calculations for math and reading proficiency levels. The state classifies assessment scores 
into five proficiency categories. The proficient category and two above-proficient categories were combined to create the "met or exceeded proficiency" category; 
two below-proficient categories were combined to create the "did not meet proficiency" category. 

Source: Authors' of student records provided by District B. 
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FIGURE 5 

Percentage of on-track and off-track freshmen in 
the 2005/06 and 2006/07 cohorts in District A who 
graduated within four years 

Percent ■ On track ■ Offtrack 




(n = 2,596) (n = 2,518) 

Note: Graduation rates are for first-time freshmen enrolled in grade 9 
at the end of the 2005/06 and 2006/07 academic years for whom on- or 
off-track status could be determined. Not included in these calculations 
were grade 9 students who died, transferred out of the district, or moved 
out of the United States in grades 10-12 and students who transferred 
into the district after grade 9 

Source: Authors' analysis of student records provided by District A. 

among students who were off track (figure 8). For 
on-track students, graduation rates ranged from 
78.0 percent to 95.8 percent, depending on student 
subgroup. For off-track students, graduation rates 
ranged from 28.6 percent to 63.6 percent. (See 
table D2 in appendix D for overall graduation rates 
for on-track and off-track students combined.) 

Gender. For female students, the on-time gradu- 
ation rate was 92.0 percent for on-track students 
and 54.0 percent for off-track students in the 
2005/06 cohort and 92.5 percent and 53.6 percent 
in the 2006/07 cohort. For male students, the on- 
time graduation rate was 89.2 percent for on-track 
students and 39.4 percent for off-track students 
in the 2005/06 cohort and 88.6 percent and 35.8 
percent in the 2006/07 cohort. 

Race/ethnicity. For White students, the on-time 
graduation rate was 95.8 percent for on-track stu- 
dents and 52.7 percent for off-track students in the 
2005/06 cohort and 94.5 percent and 52.2 percent 



in the 2006/07 cohort. For students in the other 
race/ethnicity subgroup, the on-time graduation 
rate was 84.3 percent for on-track students and 
44.9 percent for off-track students in the 2005/06 
cohort and 86.4 percent and 43.4 percent in the 
2006/07 cohort. 

IEP status. For students with an IEP, the on-time 
graduation rate was 80.6 percent for on-track stu- 
dents and 44.4 percent for off-track students in the 
2005/06 cohort and 80.2 percent and 47.0 percent 
in the 2006/07 cohort. For students without an 
IEP, the on-time graduation rate was 91.8 percent 
for on-track students and 46.4 percent for off-track 
students in the 2005/06 cohort and 91.9 percent 
and 44.2 percent in the 2006/07 cohort. 

Free or reduced-price lunch status. For stu- 
dents receiving free or reduced-price lunch, the 
on-time graduation rate was 80.9 percent for 
on-track students and 38.5 percent for off-track 
students in the 2005/06 cohort and 82.7 percent 
and 38.0 percent for off-track students in the 
2006/07 cohort. For students not receiving free 
or reduced-price lunch, the on-time graduation 
rate was 94.9 percent for on-track students and 

57.6 percent for off-track students in the 2005/06 
cohort and 94.4 percent and 57.4 percent in the 
2006/07 cohort. 

Age. For grade 9 students who were the age of typi- 
cal high school freshmen (age 14 or under at the 
start of grade 9), the on-time graduation rate was 

92.6 percent for on-track students and 53.1 percent 
for off-track students in the 2005/06 cohort and 
91.5 percent and 49.6 percent in the 2006/07 
cohort. For students who were older then the age 
of typical high school freshmen (over age 14 at 
the start of grade 9), the on-time graduation rate 
was 80.2 percent for on-track students and 28.6 
percent of off-track students in the 2005/06 cohort 
and 85.6 percent and 32.3 percent in the 2006/07 
cohort. 

Grade 8 math proficiency. For students who met 
or exceeded proficiency on the grade 8 state math 
assessment, the on-time graduation rate was 95.8 
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USING THE FRESHMAN ON-TRACK INDICATOR TO PREDICT GRADUATION IN TWO URBAN DISTRICTS 



FIGURE 6 

Percentage of on-track and off-track freshmen in the 2005/06 and 2006/07 cohorts in District A who 
graduated within four years, by student background subgroup 

Percent A. Gender ■ On track ■ Offtrack 
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F. Grade 8 reading proficiency 1 * 
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Note: Graduation rates are for first-time freshmen enrolled in grade 9 at the end of the 2005/06 and 2006/07 academic years for whom on- or off-track status 
could be determined. Grade 9 students who died, transferred out of the district, or moved out of the United States in grades 10-12 and students who trans- 
ferred into the district after grade 9 were not included in graduation calculations. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of non- 
Black minority students. 

b. Students with missing assessment scores were not included in the calculations for math and reading proficiency levels. The state classifies assessment 
scores into five proficiency categories. The proficient category and two above-proficient categories were combined to create the "met or exceeded profi- 
ciency" category; two below-proficient categories were combined to create the "did not meet proficiency" category. 

Source: Authors' analysis of student records provided by District A. 
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FIGURE 7 

Percentage of on-track and off-track freshmen in 
the 2005/06 and 2006/07 cohorts in District B who 
graduated within four years 

Percent ■ On track ■ Offtrack 




(n = 2,478) (n = 2,507) 

Note: Graduation rates are for first-time freshmen enrolled in grade 9 
at the end of the 2005/06 and 2006/07 academic years for whom on- or 
off-track status could be determined. Not included in these calculations 
were grade 9 students who died, transferred out of the district, or moved 
out of the United States in grades 10-12 and students who transferred 
into the district after grade 9. 

Source: Authors' analysis of student records provided by District B. 

percent for on-track students and 63.6 percent for 
off-track students in the 2005/06 cohort and 93.1 
percent and 52.9 percent in the 2006/07 cohort. 

For students who did not meet proficiency on the 
grade 8 math assessment, the on-time graduation 
rate was 84.6 percent for on-track students and 
41.5 percent for off-track students in the 2005/06 
cohort and 78.0 percent and 40.9 percent in the 
2006/07 cohort. 

Grade 8 reading proficiency. For students who met 
or exceeded proficiency on the grade 8 state read- 
ing assessment, the on-time graduation rate was 

94.7 percent for on-track students and 50.5 percent 
for off-track students in the 2005/06 cohort and 

93.7 percent and 52.6 percent in the 2006/07 
cohort. For students who did not meet proficiency 
on the grade 8 reading assessment, the on-time 
graduation rate was 83.3 percent for on-track stu- 
dents and 43.2 percent for off-track students in the 
2005/06 cohort and 79.3 percent and 41.5 percent 
in the 2006/07 cohort. 



Using the on-track indicator to predict 
four-year graduation rates 

The on-track indicator was a statistically sig- 
nificant predictor of on-time graduation, even 
after taking into account differences in students’ 
gender, race/ethnicity, IEP status, free or reduced- 
price lunch status (District B only), age, grade 8 
assessment test scores, and cohort. The full results 
are presented in appendix E. 

For District A, students who were on track to 
graduate at the end of their freshman year were 
6.6 times more likely to graduate than were 
students with similar characteristics who were 
off track at the end of their freshman year. For Dis- 
trict B, students who were on track at the end of 
their freshman year were 5.5 times more likely to 
graduate than were students with similar char- 
acteristics who were off track at the end of their 
freshman year. 

For both districts, the effect size, as measured by 
the changes in odds, was bigger for on-track status 
than for any student background characteristic or 
for grade 8 assessment scores. 9 This means that 
being on track to graduate at the end of freshman 
year is more predictive of a student graduating 
than racial/ethnic minority status, IEP status, or 
free or reduced-price lunch status. 



HOW THESE RESULTS COMPARE WITH 
THOSE OF OTHER STUDIES 

This section compares the results of this study 
with the findings reported in CCSR’s studies of 
Chicago students (Allensworth and Easton 2005, 
2007) and in REL Southwest’s study of five Texas 
districts (Hartman et al. 2011). All three studies 
used methods similar to the ones used in the cur- 
rent study. 



On-track rates 

The on-track graduation rates found in the current 
study are higher than those reported in the first 
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FIGURE 8 

Percentage of on-track and off-track freshmen in the 2005/06 and 2006/07 cohorts in District B who 
graduated within four years, by subgroup 



Percent A. Gender ■ On track ■ Off track 
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Note: Graduation rates are for first-time freshmen enrolled in grade 9 at the end of the 2005/06 and 2006/07 academic years for whom on- or off-track status 
could be determined. Grade 9 students who died, transferred out of the district, or moved out of the United States in grades 10-12 and students who trans- 
ferred into the district after grade 9 were not included in graduation calculations. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of non- 
Black minority students. 

b. Students with missing assessment scores were not included in the calculations for math and reading proficiency levels. The state classifies assessment 
scores into five proficiency categories. The proficient category and two above-proficient categories were combined to create the "met or exceeded profi- 
ciency" category; two below-proficient categories were combined to create the "did not meet proficiency" category. 

Source: Authors' analysis of student records provided by District B. 










HOW THESE RESULTS COMPARE WITH THOSE OF OTHER STUDIES 



13 



CCSR report (Allensworth and Easton 2005) and in 
all but one of the five Texas districts examined by 
REL Southwest (Hartman et al. 2011). In District A, 
the proportion of first-time grade 9 students classi- 
fied as on track for high school graduation within 
four years ranged from 41.1 percent in the 2008/09 
cohort to 51.5 percent in the 2005/06 cohort. In 
District B, the proportion of first-time grade 9 
students classified as on track for on-time high 
school graduation ranged from 84.6 percent in the 
2006/07 cohort to 86.9 percent in the 2009/10 co- 
hort. For Chicago students in the 1994/95-2003/04 
freshman cohorts, the on-track rate ranged from 
50 percent to 66 percent (Allensworth and Easton 
2005). For five Texas districts, the on-track rate 
ranged from 61.2 percent to 86 percent. 

In both districts, the on-track rate was higher for 
female students than for male students; for White 
students than for students of other race/ethnicity; 
for students with an IEP than for those without; 
for students who did not receive free or reduced- 
price lunch than for those who did (examined in 
District B only); for students who were age 14 or 
under at the start of freshman year than for those 
who were over age 14; and for students who met 
proficiency on grade 8 state math and reading 
assessments than for those who did not. These 
findings are consistent with those of the other two 
studies. 



Graduation rates for on-track and off-track students 

As in the CCSR and REL Southwest studies, in this 
study on-track students graduated on time at a 
higher rate than did off-track students, both over- 
all and within all student subgroups. In District A, 
the average on-time graduation rate for first-time 
freshmen who were on track at the end of grade 9 
was 2.6 times higher than the rate for students 
who were off track. In District B, the average on- 
time graduation rate for first-time freshmen who 
were on track at the end of grade 9 was 2.0 times 
higher than the rate for those who were off track. 

The difference in the graduation rate between on- 
track and off-track students was smaller in both 



The on-track graduation 
rates found in the 
current study are higher 
than those reported in 
two studies that use a 
similar methodology 



Districts A and B than 
for the Chicago students 
in the CCSR study. The 
graduation rate for 
Chicago students was 3.7 
times higher for on-track 
students than for off- 
track students in both the 
1999/2000 cohort (Allensworth and Easton 2005) 
and the 2000/01 cohort (Allensworth and Easton 
2007). In the five Texas districts examined by 
REL Southwest, the on-time graduation rate was 
1.4-3. 5 times higher for on-track students than for 
off-track students. 



The utility of an on-track indicator depends largely 
on how well it predicts on-time graduation. The 
less accurately the indicator predicts on-time 
graduation for on-track students, the greater the 
risk of identifying as on track students who need 
intervention. The less accurately the indicator 
predicts on-time graduation for off-track students, 
the greater the risk of targeting interventions to 
students who are not in need of intervention. 



The accuracy with which the on-track indicator 
predicts on-time graduation varied among the 
eight districts examined in the CCSR study, the 
REL Southwest study, and this study. For on-track 
students, the graduation rate in these districts 
ranged from 70 percent to 91 percent. For off-track 
students, the graduation rate ranged from 20 
percent to 51 percent. 

In the current study, an average of 78 percent of 
on-track students in District A and 91 percent in 
District B graduated on-time, and 39 percent of 
off-track students in District A and 45 percent in 
District B graduated on-time. In the hypothetical 
situation that both districts had used the CCSR 
on-track indicator to target interventions to off- 
track students in the cohorts examined, District 
B would have had a higher proportion of off-track 
students who graduated on time and thus might 
have been more likely to devote resources to 
students not at risk. District A would have had a 
higher proportion of on-track students who did 
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The findings suggest 
that freshman year 
academic performance 
is associated with 
graduation, over 
and above student 
background and 
prior achievement 



not graduate in four years and 
would have been more likely to fail 
to identify students at risk. 



It would benefit districts to 
consider the potential accuracy of 
the CCSR on-track indicator and 
other early warning indicators 
before adopting them as mea- 
sures to identify students at risk 
of not graduating on time. Future research could 
explore whether a more accurate indicator or set 
of indicators for predicting graduation, such as 
student attendance and study behaviors, could be 
developed for districts similar to the two districts 
in this study. 



The relationship between on-track 
status and on-time graduation 

For students in the 2005/06 and 2006/07 co- 
horts, on-track status at the end of grade 9 was 
statistically significantly associated with gradu- 
ation, after controlling for student gender, race/ 
ethnicity, IEP status, age, and math and reading 
assessment scores. Similar results were found in 
the CCSR studies. These findings are inconsistent 
with the argument that the on-track indica- 
tor predicts graduation only because it reflects 
certain aspects of student background. Rather, 
these findings suggest that freshman year aca- 
demic performance is associated with graduation, 
over and above student background and prior 
achievement. 



LIMITATIONS OF THE STUDY AND 
SUGGESTIONS FOR FUTURE RESEARCH 

Findings for other districts might not be the same 
as those from this study, whose results should be 
considered in light of several limitations. 

First, the regression models in this study used 
a limited set of indicators to control for student 
characteristics. If other student indicators were 
used, the strength of the relationship between 



on-track status and on-time graduation would 
likely differ. 

Second, differences between the two districts 
mean that the findings for each must be inter- 
preted independently. The two districts have dif- 
ferent policies for tracking students who have left 
the district, and calculated graduation rates may 
be more accurate in one district than the other 
(see appendix C). The regression models for each 
district included different student background 
variables, which limits comparability across dis- 
tricts. It may not be appropriate to compare find- 
ings from this study with similar research in other 
districts, because the findings might be limited 
by differences in district policies, practices, and 
aspects of student background examined. 

Third, students with missing assessment scores 
were not included in the regression models. Al- 
though these students were similar in background 
characteristics examined to the students who 
were included, 10 they might have differed on other 
characteristics that were not measured. 

Fourth, this study was limited to examination of 
the four-year graduation rate, defined by whether 
a student has completed all high school require- 
ments by the end of the fourth year of high school. 
It is not known whether the on-track indicator 
would predict graduation in more than four years 
as reliably as it predicts four-year graduation. 11 

Future research might examine this relationship 
in districts that differ in geographic locale and 
student body characteristics. The CCSR on-track 
indicator has not yet been studied in rural or 
suburban districts. Future research using school- 
level variables might reveal variations across 
schools in this relationship. In the CCSR study of 
Chicago students (Allensworth and Easton 2005), 
both on-track and graduation rates varied across 
schools with similar student bodies. Finally, future 
studies could compare the on-track indicator with 
other early warning indicators or explore how the 
indicator might be integrated into early warning 
systems that include multiple indicators. 
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APPENDIX A 

STUDY DATA AND ANALYSIS 

This appendix describes the study data and 
methodology. 



Data sources 

Both districts in this study provided student-level 
data for the freshman cohorts of 2005/06, 2006/07, 
2007/08, and 2008/09. District B also provided 
data for the 2009/10 freshman cohort; data for the 
2009/10 cohort were not available from District A. 

The data files included the following information 
for each student: district entry date and (if applica- 
ble) exit date and exit code, course records (includ- 
ing grades and credits earned), birth date, gender, 
race/ethnicity, individualized education program 
(IEP) status, limited English proficiency (LEP) 
status, and grade 8 scores and proficiency levels 
on state math and reading assessments. Because 
analyses excluded repeat freshmen (grade 9 stu- 
dents whose unique identifier number was found in 
the grade 9 roster file of a previous school year), the 
districts also provided student roster data for the 
2004/05 freshman cohort so that repeat freshmen 
in 2005/06 could be identified. Repeat freshmen 
were excluded because the Chicago Consortium 
for School Research (CCSR) on-track indicator is 
based on research on first-time freshmen. For the 
2005/06 and 2006/07 cohorts, the district files in- 
cluded graduation data and information on student 
withdrawals over the course of high school. District 
B files also indicated whether each student received 
free or reduced-price lunch during grade 9. (These 
data were not made available for District A.) To 
protect student privacy, an encrypted identifier was 
used to match students across data files. 

Definitions of on-track and off-track status. Stu- 
dents were identified as on track if they met the 
following two criteria by the end of grade 9: 

• They had accumulated the number of credits 
needed for promotion to grade 10, as defined 
by district policy. 



• They had received no more than one failing 
grade for a semester in a core subject (English, 
math, science, or social studies). 

Students were identified as off track at the end of 
the first year of high school if either of these crite- 
ria was not met. 

Definition of on-time graduation status. Students 
in the 2005/06 and 2006/07 cohorts were classified 
as having graduated on time if district files identi- 
fied them as having graduated on a date that was 
within four years of the date they entered grade 9. 
Additional information appears in appendix B. 



Analytic samples 

An analytic sample was constructed for each 
research question. For the first question, which 
addresses freshman on-track and off-track rates, 
the analytic sample for each cohort included all 
students enrolled in the district at the end of their 
first freshman year, with the exception of students 
with missing or problematic enrollment data and 
students for whom only one semester of course 
data was available. (On-track status could not be 
determined for these students.) For the second 
research question, which addresses graduation 
rates by freshman on-track status, students were 
excluded from the sample if they had died, moved 
outside the United States, or transferred out of the 
district to another school district, a state-approved 
alternative education program, or home school. It 
is unknown whether these students graduated on 
time. 

For the third research question, which addresses 
the extent to which the on-track indicator pre- 
dicts graduation after controlling for student 
background variables, additional students were 
excluded from the sample if their grade 8 math or 
reading assessment test score was not available. 

The analytic samples were similar across research 
questions and cohorts in the distribution of stu- 
dent characteristics. Appendix C includes detailed 
information on the construction of the analytic 
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Analysis 



samples and the distribution of student character- 
istics across those samples. 



For the first research questions, the percentages 
of students who were on track and off track at the 
end of their first year of high school were calcu- 
lated for each of the four or five cohorts in each 
district. These percentages were calculated overall 
and for student background subgroups based on 
gender, race/ethnicity, IEP status, age at the start 
of grade 9, and proficiency level on grade 8 math 
and reading assessments. The number of students 
in some subgroups was small enough to risk 
disclosing the identities of the sample districts and 
to produce potentially misleading results (such as 
oscillations in on-track rates across cohorts that 
might be interpreted as large but that were attrib- 
utable to a small number of students). For these 
reasons, student data on English language learner 
status were not analyzed, and all racial/ethnic 



minority students were included in one “other 
race/ethnicity” subgroup instead of analyzing each 
racial/ethnic subgroup separately. 

For the two cohorts in each district included in the 
second research question, the percentages of on- 
track and off-track students who graduated within 
four years of starting high school were calculated 
overall and for student background subgroups. 

For the two cohorts in each district studied for the 
third research question, the relationship between 
the on-track indicator and the probability of 
student graduation was examined, controlling for 
student background characteristics and for grade 8 
math and reading assessment scores. A hierarchi- 
cal logistic model was used for the analysis, which 
was appropriate because of the binary outcome 
(graduated on time or did not graduate on time) 
and because of the hierarchical structure of the 
data, with students nested in schools. Details on 
the model specifications are in appendix E. 



APPENDIX B. DETERMINATION OF ON-TRACK STATUS AND GRADUATION STATUS 



17 



APPENDIX B 

DETERMINATION OF ON-TRACK STATUS 
AND GRADUATION STATUS 

On-track status and on-time graduation were 
determined using the Chicago Consortium for 
School Research (CCSR)’s methodology (see 
Allensworth and Easton 2005). 



Determining on-track status 

Student data files contained information on the 
courses for which students were registered, the 
semester when each course was taken, the letter 
grade earned for each course, and the number of 
credits earned for each course. Students were clas- 
sified as on track if, during their first year of high 
school, they had earned enough course credits 
to be promoted to grade 10 and if they earned 



no more than one semester F in a core subject 
(English, math, science, or social studies). District 
A required five full-year credits to be promoted to 
grade 10. District B required four full-year credits 
to be promoted for the 2005/06, 2006/07, and 
2007/08 cohorts and five full-year credits for the 
2008/09 and 2009/10 cohorts. 



Determining on-time graduation 

Students were identified as graduates if the 
district files included a graduation date for them. 
They were identified as on-time graduates if the 
graduation date was no more than four years from 
the start of grade 9. Neither district recognizes 
individuals earning a General Educational Devel- 
opment (GED) diploma as high school graduates. 
GED information was not included in the district 
files. 
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APPENDIX C 
ANALYTIC SAMPLES 

This appendix describes how the analytic samples 
were created for each research question and com- 
pares the distribution of student characteristics 
across analytic samples. 



Building the analytic samples 

The steps followed to exclude students from the 
analytic sample for each of the three research 
questions are presented in tables Cl (District A) 
and C2 (District B). For each cohort, the tables 
present the number of students excluded in each 



TABLE Cl 

Analytic sample exclusions for each cohort of students in District A 



Reason for exclusion 


2005/06 


2006/07 


2007/08 


2008/09 


Total number of grade 9 students in the district file 


4,452 


4,556 


4,209 


4,295 


Step 1 Students who were not enrolled at the end of grade 9 


818 


991 


431 


374 


Step 2 Students with missing or problematic enrollment data 


19 


12 


0 


0 


Step 3 Repeat freshmen 


579 


675 


1,017 


1,061 


Step 4 Students for whom on-track or off-track status could not be determined 
at the end of grade 9 


95 


68 


95 


89 


Remaining sample: number of students included in analysis for research question 1 


2,941 


2,810 


2,666 


2,771 


Step 5 Students who died, moved out of the United States, or transferred out of 

the district in grades 10-12 


344 


292 


na 


na 


Remaining sample: number of students included in analysis for research question 2 


2,596 


2,518 


na 


na 


Step 6 Students with missing grade 8 assessment test data 3 


591 


529 


na 


na 


Remaining sample: number of students included in analysis for research question 3 
(hierarchical linear model analysis) 


2,005 


1,989 


na 


na 



na is not applicable: freshmen in these cohorts had not yet completed four years of high school and so were not included in the analysis for the first two 
research questions. 

a. In the hierarchical linear model analysis, students with missing data were dropped casewise. 

Source: Authors' analysis of student records provided by District A. 



TABLE C2 

Analytic sample exclusions for each cohort of students in District B 



Reason for exclusion 


2005/06 2006/07 2007/08 2008/09 2009/10 


Total number of grade 9 students in the district file 


3,340 


3,452 


3,415 


3,685 


3,487 


Step 1 Students who were not enrolled at the end of freshman year 


321 


359 


308 


483 


319 


Step 2 Students with missing or problematic enrollment data 


59 


59 


24 


0 


3 


Step 3 Repeat freshmen 


74 


134 


125 


94 


120 


Step 4 Students for whom on- or off-track status could not be determined at the 
end of grade 9 


109 


102 


105 


79 


76 


Remaining sample: number of students included in analysis for research question 1 


2,777 


2,798 


2,853 


3,029 


2,969 


Step 5 Students who died, moved out of the United States, or transferred out of 
the district in grades 10-12 


299 


291 


na 


na 


na 


Remaining sample: number of students included in analysis for research question 2 


2,478 


2,507 


na 


na 


na 


Step 6 Students with missing grade 8 assessment test data 3 


360 


322 


na 


na 


na 


Remaining sample: number of students included in analysis for research question 3 
(hierarchical linear model analysis) 


2,118 


2,185 


na 


na 


na 



na is not applicable: freshmen in these cohorts had not yet completed four years of high school and so were not included in the analysis for the first two 
research questions. 

a. In the hierarchical linear model analysis, students with missing data were dropped casewise. 

Source: Authors' analysis of student records provided by District B. 
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step and the number of students in the final ana- 
lytic sample. 



Research Question 1 samples 

The data files received from the districts included 
all students who enrolled in grade 9 at any point 
and for any length of time during the 2005/06- 
2008/09 academic years for District A and the 
2005/06-2009/10 academic years for District B. 
Based on input from the districts and in keeping 
with the sample decisions made in prior studies 
(Allensworth and Easton 2005; Hartman et al. 
2011), the following students were excluded from 
the analytic samples for research question 1: stu- 
dents who were not enrolled at the end of freshman 
year (step 1), students for whom enrollment status 
could not be confirmed because of missing data 
(step 2), and students who were repeat freshmen 
(step 3). Consistent with Hartman et al. (2011), 
students enrolled at the end of grade 9 for whom 
on-track status could not be determined because of 
incomplete course records were also excluded. 

Step 1 

District A. To identify students in District A who 
had left the district in grade 9, students whose 
second semester course records were missing were 
identified. (Leave dates alone were not reliable 
indicators of whether a student had left the district 
during grade 9: some students with leave dates in 
grade 9 had left the district and returned before 
the end of grade 9, and dates of re-entry were not 
noted in the district files.) To verify that students 
with missing second semester course data had left 
the district, the study team examined whether 
any leave code had been assigned to them by the 
district. Across the five cohorts, 80-90 percent of 
these students had leave codes assigned to them. 
The remaining 10-20 percent had no grade 9 leave 
codes. However, the group without leave codes was 
similar to the group with leave codes in that less 
than 5 percent of these students were enrolled in 
the district in grade 12. This suggests, if indirectly, 
that students missing grade 9 second semester 
course records had left the district, whether leave 



codes had been assigned to them or not. Based on 
this analysis, all students without grade 9 second 
semester course grades were classified as “not 
enrolled at the end of freshman year,” and these 
students were removed from the analytic sample. 

District B. For District B, all students who had a 
leave code indicating that they had left the district 
and who had a leave date before the end of their 
freshman year were classified as “not enrolled 
at the end of freshman year” and removed from 
the analytic sample. Leave codes and dates were 
crosschecked with student course records to verify 
that no second semester course data were available 
for these students. 

Step 2. For both districts, there were students 
in each cohort whose birthdates, entry dates, or 
leave dates suggested that they might not have 
been freshmen in the district and cohort in which 
they were included. Examples included having a 
birthdate indicating that the student was not yet of 
school age or having a district entry date after the 
end of grade 9. These students were removed from 
the analytic sample, as were students whose birth- 
dates, entry dates, and leave dates were all missing. 

Step 3. For both districts, repeat freshmen were 
identified by comparing the grade 9 student roster 
for each cohort with the grade 9 roster for the pre- 
vious year. If a student identifier number appeared 
in both rosters, the student was classified as a 
repeat freshman and excluded from the sample. 

Step 4. For both districts, students missing grade 9 
first semester course data were excluded from the 
sample. 



Research question 2 samples 

The exclusions to the analytic samples for research 
question 2 (steps 5-7) are consistent with Hart- 
man et al. (2011). 

Step 5. For both districts, graduation rate calcula- 
tions excluded students whose leave code indicated 
that they had died in grades 10-12. 
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For both districts, graduation rate calculations 
excluded students whose leave code indicated 
that they moved out of the United States in grades 
10-12. These students may or may not have gradu- 
ated on time. 

For both districts, graduation rate calculations 
excluded students whose leave code indicated 
that they had transferred to another district or an 
approved alternative education program in grades 
10-12. These students may or may not have gradu- 
ated on time. 

District A classifies students as having transferred 
to another educational setting only if a transcript 
request from the new school or district was on 
file. Because District B does not have a similar 
method for confirming student transfers, it is 
possible that some District B students excluded 
in step 6 did not transfer but dropped out. If so, 
the District B graduation rates calculated in this 
study might be higher than actual graduation 
rates. Representatives of District B acknowledged 
this possibility but maintained that student leave 
codes are nevertheless the best gauge of whether 
students transferred to another educational 
setting. 



Research question 3 sample 

Step 6. The hierarchical logistic regression analysis 
used to address research question 3 used casewise 
deletion to drop students with missing data. 12 For 



both districts, the only variables with missing data 
were math and reading assessment test scores. 

One concern related to the casewise deletion of 
students with missing assessment test scores was 
the large number of students dropped from the 
analysis for this reason (shown in the row labeled 
step 7 in tables Cl and C2). To see the effect of 
these exclusions, the results from the final model 
(appendix E) were compared with the results 
from a model that included assessment test scores 
as variables (so that students with missing test 
scores could be included in the sample). Because 
the models that excluded students with missing 
assessment test scores produced a bigger (less 
conservative) estimated effect for the on-track 
indicator than the models that included them, as- 
sessment test scores were included in the model. 

Another concern related to casewise deletion was 
that certain groups of students might be dispro- 
portionately excluded from the sample. As shown 
in tables C3 and C4, however, the samples for 
research question 3 were similar to the samples for 
research questions 1 and 2 in the distribution of 
student characteristics. 



Statistical descriptions of the analytic samples 

Comparing the distribution of student charac- 
teristics across analytic samples shows, for both 
districts, a similar distribution of student charac- 
teristics across cohorts and research questions. 
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TABLE C3 

Distribution of grade 9 students in the District A study samples, by research question and student subgroup, 2005/06-2008/09 









2005/06 










2006/07 






2007/08 


2008/09 


2009/10 


Student 


Question 1 


Question 2 


Question 3 


Question 1 


Question 2 


Question 3 


Question 1 


Question 1 


Question 1 


subgroup 


Number Percent Number Percent Number Percent Number Percent Number Percent Number Percent Number Percent Number Percent Number Percent 


Complete sample 


2,941 


100 


2,596 


100 


2,006 


100 


2,810 


100 


2,518 


100 


1,989 


100 


2,666 


100 


2,771 


100 


2,337 


100 


Gender 


Female 


1,499 


51 


1,342 


51.7 


1,059 


52.8 


1,437 


51.1 


1,299 


51.6 


1,034 


52 


1,335 


50.1 


1,398 


50.5 


1,183 


50.6 


Male 


1,442 


49 


1,254 


48.3 


947 


47.2 


1,373 


48.9 


1,219 


48.4 


955 


48 


1,331 


49.9 


1,373 


49.5 


1,153 


49.3 


Race/ethnicity 


White 


627 


21.3 


536 


20.6 


433 


21.6 


616 


21.9 


546 


21.7 


447 


22.5 


518 


19.4 


554 


20 


560 


24 


Other race/ethnicity 3 


2,314 


78.7 


2,060 


79.4 


1,573 


78.4 


2,194 


78.1 


1,972 


78.3 


1,542 


77.5 


2,148 


80.6 


2,217 


80 


1,777 


76 


Program participation 


Students with an individu- 
alized education program 


672 


22.8 


583 


22.5 


437 


21.8 


598 


21.3 


520 


20.7 


402 


20.2 


577 


21.6 


582 


21 


430 


18.4 


Age at the start of freshman year 


14 or under 


1,822 


62 


1,649 


63.5 


1,313 


65.5 


1,781 


63.4 


1,634 


64.9 


1,322 


66.5 


1,679 


63 


1,710 


61.7 


2,212 


94.7 


Over 14 


1,119 


38 


947 


36.5 


693 


34.5 


1,029 


36.6 


884 


35.1 


667 


33.5 


987 


37 


1,061 


38.3 


125 


5.3 


Grade 8 proficiency level on state math assessment 13 


Proficiency data missing 


682 


23.2 


581 


22.4 


na 


na 


638 


22.7 


525 


20.8 


na 


na 


673 


25.2 


777 


28.0 


512 


21.9 


Proficiency data available 


2,259 


76.8 


2,015 


77.6 


2,006 


100 


2,172 


77.3 


1,993 


79.2 


1,989 


100 


1,993 


74.8 


1,994 


72.0 


1,825 


78.1 


Met or exceeded 
proficiency c 


884 


30.1 


812 


31.3 


812 


40.5 


1,147 


40.8 


1,073 


42.6 


1,073 


53.9 


1,128 


42.3 


1,069 


38.6 


1,002 


42.9 


Did not meet proficiency 11 


1,375 


46.7 


1,203 


46.3 


1,194 


59.5 


1,025 


36.5 


920 


36.6 


916 


46.1 


865 


32.5 


925 


33.4 


823 




Grade 8 proficiency level on state reading assessment 13 


Proficiency data missing 


690 


23.5 


585 


22.5 


na 


na 


637 


22.7 


525 


20.8 


na 


na 


671 


25.2 


786 


28.4 


524 


22.4 


Proficiency data available 


2,251 


76.5 


2,011 


77.5 


2,006 


100 


2,173 


77.3 


1,993 


79.2 


1,989 


100 


1,995 


74.8 


1,985 


71.6 


1,813 


77.6 


Met or exceeded 
proficiency c 


1,484 


50.4 


1,344 


51.8 


1,343 


66.9 


1,313 


46.7 


1,221 


48.5 


1,221 


61.4 


1,360 


51 


1,297 


46.8 


1,102 


47.2 


Did not meet proficiency 13 


767 


26.1 


667 


25.7 


663 


33.1 


860 


30.6 


772 


30.7 


768 


38.6 


635 


23.8 


688 


24.8 


711 


30.4 



na is not applicable: students with missing data were excluded from the analysis. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of non-Black minority students. 

b. The state classifies assessment test scores into five proficiency categories. The proficient category and two above-proficient categories were combined to create the "met or exceeded proficiency" category; two 
below-proficient categories were combined to create the "did not meet proficiency" category. 

c. As a percentage of students in the sample for whom proficiency data were available. 

Source: Authors' analysis of student records provided by District A. 
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TABLE C4 

Distribution of grade 9 students in the District B study samples by research question and student subgroup, 2005/06-2009/10 



2005/06 2006/07 2007/08 2008/09 2009/10 



Student 


Question 1 


Question 2 


Question 3 


Question 1 


Question 2 


Question 3 


Question 1 


Question 1 


Question 1 


subgroup 


Number Percent Number Percent Number Percent Number Percent 


Number Percent Number Percent 


Number Percent Number Percent Number Percent 


Complete sample 


2,777 


100 


2,478 


100 


2,118 


100 


2,798 


100 


2,507 


100 


2,185 


100 


2,853 


100 


3,029 


100 


2,969 


100 


Gender 3 


Female 


1,394 


50.2 


1,250 


50.4 


1,078 


50.9 


1,372 


49.0 


1,245 


49.7 


1,100 


50.3 


1,420 


49.8 


1,439 


47.5 


1,445 


48.7 


Male 


1,383 


49.8 


1,228 


49.6 


1,040 


49.1 


1,426 


51.0 


1,262 


50.3 


1,085 


49.7 


1,433 


50.2 


1,590 


52.5 


1,524 


51.3 


Race/ethnicity 


White 


1,388 


50.0 


1,277 


51.5 


1,147 


54.2 


1,274 


45.5 


1,187 


47.3 


1,074 


49.2 


1,277 


44.8 


1,319 


43.5 


1,265 


42.6 


Other race/ethnicity 3 


1,389 


50.0 


1,201 


48.5 


971 


45.8 


1,524 


54.5 


1,320 


52.7 


1,111 


50.8 


1,576 


55.2 


1,710 


56.5 


1,704 


57.4 


Program participation 


Has an individualized 
education program 


322 


11.6 


261 


10.5 


215 


10.2 


389 


13.9 


324 


12.9 


275 


12.6 


409 


14.3 


447 


14.8 


424 


14.3 


Identified as receiving 
free or reduced-price 
lunch 


1,006 


36.2 


845 


34.1 


676 


31.9 


1,102 


39.4 


931 


37.1 


760 


34.8 


1,162 


40.7 


1,390 


45.9 


1,447 


48.7 


Identified as special 
education 


30 


1.1 


20 


0.8 


17 


0.8 


42 


1.5 


37 


1.5 


32 


1.5 


81 


2.8 


125 


4.1 


131 


4.4 


Identified as an English 
language learner 


0 


0.0 


0 


0.0 


0 


0.0 


30 


1.1 


29 


1.2 


10 


0.5 


43 


1.5 


71 


2.3 


113 


3.8 


Age at the start of freshman year 


14 or under 


2,286 


82.3 


2,056 


83.0 


1,783 


84.2 


2,277 


81.4 


2,066 


82.4 


1,826 


83.6 


2,397 


84 


2,583 


85.3 


2,531 


85.2 


Over 14 


491 


17.7 


422 


17.0 


335 


15.8 


521 


18.6 


441 


17.6 


359 


16.4 


456 


16 


446 


14.7 


438 


14.8 


Grade 8 proficiency level on state math assessment 1 
































Proficiency data missing 


444 


16.0 


348 


14.0 


na 


na 


393 


14.0 


320 


12.8 


na 


na 


398 


14.0 


257 


8.5 


252 


8.5 


Proficiency data available 


2,333 


84.0 


2,130 


86.0 


2,118 


100 


2,405 


86.0 


2,187 


87.2 


2,185 


100 


2,455 


86.0 


2,772 


91.5 


2,717 


91.5 


Met or exceeded 
proficiency c 


1,324 


47.7 


1,258 


50.8 


1,252 


59.1 


1,945 


69.5 


1,808 


72.1 


1,808 


82.7 


2,127 


74.6 


2,389 


78.9 


2,349 


79.1 


Did not meet proficiency c 


1,009 


36.6 


872 


35.2 


866 


40.9 


460 


16.4 


379 


15.1 


377 


17.3 


328 


11.5 


383 


12.6 


368 


12.4 


Grade 8 proficiency level on state reading assessment 13 


Proficiency data missing 


454 


16.3 


356 


14.4 


0 


0 


394 


14.1 


321 


12.8 


0 


0 


398 


14.0 


279 


9.2 


263 


8.9 


Proficiency data available 


2,323 


83.7 


2,122 


85.6 


2,118 


100 


2,402 


85.9 


2,186 


87.2 


2,185 


100 


2,455 


86.0 


2,750 


90.8 


2,706 


91.1 


Met or exceeded 
proficiency c 


1,613 


58.1 


1,508 


60.9 


1,505 


71.1 


1,839 


65.7 


1,693 


67.5 


1,693 


77.5 


1,988 


69.7 


2,249 


74.2 


2,305 


77.6 


Did not meet 
proficiency c 


710 


25.6 


614 


24.8 


613 


28.9 


565 


20.2 


493 


19.7 


492 


22.5 


467 


16.4 


501 


16.5 


401 


13.5 



na is not applicable: students with missing data were excluded from the analysis. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of non-Black minority students. 

b. The state classifies assessment test scores into five proficiency categories. The proficient category and two above-proficient categories were combined to create the "met or exceeded proficiency" category; two 
below-proficient categories were combined to create the "did not meet proficiency" category. 

c. As a percentage of students in the sample for whom proficiency data were available. 

Source: Authors' analysis of student records provided by District B. 
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APPENDIX D 

GRADUATION RATES FOR FRESHMEN IN 
THE 2005/06 AND 2006/07 COHORTS 

This appendix presents overall graduation rates for 
on-track and off-track students combined. 



TABLE D1 

Percentage of freshmen in the 2005/06 and 2006/07 cohorts in District A who graduated within four years, 
overall and by student background subgroup 







2005/06 






2006/07 




Student subgroup 


Sample 


Graduated 
on time 


Percent of 
sample 


Sample 


Graduated 
on time 


Percent of 
sample 


Complete sample 


2,596 


1,464 


56.4 


2,518 


1,335 


53.0 


Gender 


Female 


1,342 


829 


61.8 


1,299 


767 


59.0 


Male 


1,254 


635 


50.6 


1,219 


568 


46.6 


Race/ethnicity 


White 


536 


376 


70.1 


546 


357 


65.4 


Other race/ethnicity 3 


2,060 


1,088 


52.8 


1,972 


978 


49.6 


Program participation 


Identified as having an individualized education program 


583 


283 


48.5 


520 


212 


40.8 


Age at the start of freshman year 


14 or under 


1,649 


1,087 


65.9 


1,634 


1,018 


62.3 


Over 14 


947 


377 


39.8 


884 


317 


35.9 


Grade 8 proficiency level on state math assessment 


Proficiency data missing 


581 


267 


46.0 


525 


239 


45.5 


Proficiency data available 


2,015 


1,197 


59.4 


1,993 


1,096 


55.0 


Met or exceeded proficiency 11 


812 


651 


80.2 


1,073 


753 


70.2 


Did not meet proficiency c 


1,203 


546 


45.4 


920 


343 


37.3 


Grade 8 proficiency level on state reading assessment 


Proficiency data missing 


585 


268 


45.8 


525 


241 


45.9 


Proficiency data available 


2,011 


1,196 


59.5 


1,993 


1,094 


54.9 


Met or exceeded proficiency 11 


1,344 


929 


69.1 


1,221 


833 


68.2 


Did not meet proficiency c 


667 


267 


40.0 


772 


261 


33.8 



Note: Graduation rates are for first-time freshmen enrolled in grade 9 at the end of the 2005/06 and 2006/07 academic years for whom on-track or off-track 
status could be determined. Students who died, transferred out of the district, or moved out of the United States in grades 10-12 and students who trans- 
ferred into the district after grade 9 were not included in graduation calculations. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of 
non-Black minority students. 

b. The state classifies assessment test scores into five proficiency categories. The proficient category and two above-proficient categories were combined to 
create the "met or exceeded proficiency" category; two below-proficient categories were combined to create the "did not meet proficiency" category. 

c. As a percentage of students in the sample for whom proficiency data were available. 

Source: Authors' analysis of student records provided by District A. 
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TABLE D2 

Percentages of freshmen in 2005/06 and 2006/07 cohorts in District B who graduated within four years, 
overall and by subgroup classification 







2005/06 






2006/07 




Student subgroup 


Sample 


Graduated 
on time 


Percent of 
sample 


Sample 


Graduated 
on time 


Percent of 
sample 


Complete sample 


2,478 


2,114 


85.3 


2,507 


2,118 


84.5 


Gender 


Female 


1,250 


1,098 


87.8 


1,245 


1,087 


87.3 


Male 


1,228 


1,016 


82.7 


1,262 


1,031 


81.7 


Race/ethnicity 


White 


1,277 


1,192 


93.3 


1,187 


1,092 


92.0 


Other race/ethnicity 3 


1,201 


922 


76.8 


1,320 


1,026 


77.7 


Program participation 


Identified as having an individualized education program 


261 


194 


74.3 


324 


238 


73.5 


Identified as receiving free or reduced-price lunch 


845 


608 


72.0 


931 


673 


72.3 


Identified as an English language learner 


0 


0 


0.0 


29 


21 


72.4 


Age at the start of freshman year 


14 or under 


2,056 


1,819 


88.5 


2,066 


1,790 


86.6 


Over 14 


422 


295 


69.9 


441 


328 


74.4 


Grade 8 proficiency level on state math assessment 11 


Proficiency data missing 


348 


272 


78.2 


320 


244 


76.3 


Proficiency data available 


2,130 


1,842 


86.5 


2,187 


1,874 


85.7 


Met or exceeded proficiency 11 


1,258 


1,188 


94.4 


1,808 


1,621 


89.7 


Did not meet proficiency c 


872 


654 


75.0 


379 


253 


66.8 


Grade 8 proficiency level on state reading assessment 13 


Proficiency data missing 


356 


278 


78.1 


321 


246 


76.6 


Proficiency data available 


2,122 


1,836 


86.5 


2,186 


1,872 


85.6 


Met or exceeded proficiency 0 


1,508 


1,386 


91.9 


1,693 


1,530 


90.4 


Did not meet proficiency 0 


614 


453 


73.8 


493 


342 


69.4 



Note: Graduation rates are for first-time freshmen enrolled in grade 9 at the end of the 2005/06 and 2006/07 academic years for whom on-track or off-track 
status could be determined. Students who died, transferred out of the district, or moved out of the United States in grades 10-12 and students who trans- 
ferred into the district after grade 9 were not included in graduation calculations. 

a. Includes Asian, Black, Hispanic, Native American, and multiracial students, who were grouped into a single category because of the small numbers of 
non-Black minority students. 

b. The state classifies assessment test scores into five proficiency categories. The proficient category and two above-proficient categories were combined to 
create the "met or exceeded proficiency" category; two below-proficient categories were combined to create the "did not meet proficiency" category. 

c. As a percentage of students in the sample for whom proficiency data were available. 

Source: Authors' analysis of student records provided by District B. 
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APPENDIX E 

REGRESSION MODEL SPECIFICATIONS 
AND RESULTS 

This appendix describes the regression model used 
in the study and discusses the results. 



Model specifications 

To address research question 3 on how well the on- 
track indicator predicts four-year graduation rates, 
a two-level (students within schools) hierarchical 
generalized linear model with binary outcomes 
(Raudenbush and Bryk 2002) was used to examine 
the relationship between the on-track indica- 
tor and on-time student graduation, controlling 
for certain student background characteristics. 
This type of model is a special case of hierarchi- 
cal linear model in which the outcome is binary 
rather than continuous and takes into account the 
hierarchical nature of the data. 

Level 1 of the District A model included student 
measures related to cohort, on-track status, 
gender, race/ethnicity, individualized education 
program (IEP) status, age, and grade 8 math and 
reading assessment test scores. Level 1 of the Dis- 
trict B model included student measures related to 
cohort, on-track status, gender, race/ethnicity, free 
or reduced-price lunch status, age, and grade 8 
math and reading assessment test scores. 

Addressing research question 3 entailed examin- 
ing the extent to which these level 1 (student-level) 
predictors rather than level 2 (school-level) predic- 
tors account for differences in on-time graduation. 
Because level 1 estimates could be biased by the 
omission of level 2 predictors, specification tests 
are conducted to check whether any of the student- 
level predictors were vulnerable to omitted vari- 
able bias of this type (Raudenbush and Bryk 2002, 
p. 262). For both districts, these tests did not yield 
any evidence of such bias, so no level 2 predictors 
were included in the models. 13 



across schools, significance tests and likeli- 
hood ratio tests were used to identify significant 
between-school variation in these slopes. For 
District A, all level 1 coefficients, except that as- 
sociated with cohort, had between-school varia- 
tions in slope that were not significantly different 
from zero. Therefore, these covariates were fixed at 
level 1. The coefficient associated with cohort var- 
ied significantly across schools and was therefore 
specified as varying at level 2. For District B, none 
of the level 1 coefficients showed significant varia- 
tion between schools and therefore were specified 
as fixed. 

A preliminary analysis included an interaction 
term (on-track indicator * cohort) to examine 
whether the association between the on-track 
indicator and on-time graduation varied across 
the two cohorts. The interaction term was nonsig- 
nificant and was therefore dropped from the final 
model, allowing more straightforward interpreta- 
tion of the model results. 



In the analysis, the binary outcome Y took a 
value of 1 if a student graduated on time and 0 
otherwise. The probability of on-time graduation, 
(iy = P(Yjj = 1) for student i in school; was trans- 
formed using the logit link. The logit is the log of 
the odds of the event (on-time graduation), with 
the odds defined as the probability of the event 
divided by one minus the probability of the event: 



lij = log 



.1 Uij J 



The natural logarithm e was used as the base of 
the logarithm function. The above transformed 
variable was then modeled as the outcome in the 
two-level models described below. 



District A 

Level 1, students-within-schools 

*lij - n oj + <Pj c ij + djTij + In p pC pij + e { j 



To determine whether the student-level coef- 
ficients should be specified as fixed or varying 



where n 0j is the regression coefficient; c, ; is the 
cohort of student i in school; (0 = 2005/06 and 
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1 = 2006/07); T » is the freshman on-track sta- 
tus of student i in school; (0 = off track and 
1 = on track); X pij is the value of characteristic 
p for student i in school; (gender: 0 = male and 
1 = female; race/ethnicity: 0 = White and 1 = other 
race/ethnicity; IEP status: 0 = no IEP and 1 = IEP; 
age at start of freshman year: 0 = age 14 or under 
and 1 = over age 14; standardized grade 8 math as- 
sessment test score; standardized grade 8 reading 
assessment score); and is a random error term 
for each student. 

Math and reading assessment test scores were 
grand mean-centered to allow for easier interpre- 
tation of coefficients. 

Level 2, schools 

n 0j — Poo + r o j 
<Pj = Yoi + r 0j 
6; = Y02 
71 pj = Yo3 

where (3 00 is the average log-odds of graduating 
across schools, y 01 is the average cohort effect for 
students who are not on track (adjusted for their 
baseline characteristics), and r 0; is a school-level 
random error term. 

District B 

Level 1, students-within-schools 

hij - n oj + <Pj c ij + QjTij + ln p jX- P ij + Sy 

where n {)J is the regression coefficient; c t] is the 
cohort of student i in school; (0 is 2005/06 and 1 
is 2006/07); 7', ; is the freshman on-track status of 
student i in school; (0 is off track and 1 is on track); 
X pi j is the value for characteristic p for student 
i in school; (gender: 0 = male and 1 = female; 

Asian ethnicity: 0 = White and 1 = Asian; Black or 
ffispanic ethnicity: 0 = White and 1 = Black or His- 
panic; free or reduced-price lunch status: 0 = did 
not receive and 1 = received; age at start of fresh- 
man year: 0 = age 14 or under and 1 = over age 14); 
standardized grade 8 math assessment test score; 



standardized grade 8 reading assessment score); 
and gy is a random error term for each student. 

Math and reading assessment test scores were 
grand mean-centered to allow for easier interpre- 
tation of coefficients. 

Level 2, schools 

j = Poo + r o j 
<Pj = y oi 
0j = Yo 2 
71 pj = Y03 

where p oo is the average log-odds of graduating 
across schools, y 01 is the average cohort effect for 
students who are not on track (adjusted for their 
baseline characteristics), and r 0; is a school-level 
random error term. 

The analytic sample for District A included 3,994 
students from two cohorts in 17 schools. School 
size averaged 235 students and ranged from 46 
students to 638 students. The analytic sample 
for District B included 4,303 students from two 
cohorts in 7 schools. School size averaged 615 
students and ranged from 21 students to 1,122 
students. Statistical descriptions of the samples are 
presented in tables C3 and C4 in appendix C. 



Tables El and E2 present the coefficient estimates 
with standard errors and the odds ratios for the 
variable in the logistic regression models. Each 
coefficient represents an estimate of the degree 
to which the log-odds of graduating on time is 
associated with each predictor, holding all other 
predictors constant. Odds ratios indicate the 
multiplicative change in odds associated with each 
unit increase in the predictor (a change from 0 to 
1 for a dummy variable or an increase of one stan- 
dard deviation for assessment test scores), holding 
constant the values of all other predictors. 

The regression analysis showed a statistically 
significant relationship (at the 0.001 significance 
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level) between on-track status and the probability 
of graduating within four years, controlling for 
student background variables and cohort. Students 
who were on track at the end of their freshman 
year graduated at higher rates than students who 
were off track. 

For District A, the regression analysis also re- 
vealed associations between on-time graduation 
and each student background variable, holding 
constant all other variables (see table El). Students 
were more likely to graduate if they were female, 
were in the other race/ethnicity subgroup, had 
an IEP, 14 were age 14 or under, and scored higher 
on the grade 8 math or reading assessment. The 
relationship between graduation and each of these 
variables was significant at the 0.01 level. On-time 
graduation was not found to have a significant as- 
sociation with cohort. 

For District B, the regression analysis revealed 
associations between on-time graduation and 
some of the student background variables, holding 



constant all other variables (see table E2). Students 
were more likely to graduate if they were female, 
were White, did not receive free or reduced-price 
lunch, were age 14 or under, and scored higher 
on the grade 8 math assessment. The relationship 
between graduation and each of these variables 
was significant at the 0.001 level. On-time gradu- 
ation was not found to have a significant associa- 
tion with cohort, Asian (as compared with White) 
ethnicity, or having scored higher on the grade 8 
reading assessment. 

The odds ratios in tables El and E2 indicate how 
many times higher the estimated odds of on-time 
graduation are with each unit increase in the 
predictor (a change from 0 to 1 for a dichotomous 
variable or an increase of one standard deviation for 
assessment test scores), holding constant the values 
of all other predictors. For the on-track indicator, 
the odds ratio estimated by the regression model 
was 6.6 for District A and 5.5 for District B. For 
both districts, the odds ratio for on-track status was 
larger than the odds ratios for all other predictors. 



TABLE El 

Regression analysis results for District A 



Predictor 


Estimated 

coefficient 


Standard 

error 


Odds 

ratio 


Odds ratio: 
95 percent 
confidence 
interval 


On-track 


•j g*** 


0.1 


6.6 


5. 5-7.9 


Female 


0.3*” 


0.1 


1.4 


1.2-1 .6 


Other race/ethnicity 


0.7”* 


0.1 


1.9 


1.5-2. 5 


Identified as having an individualized education program 


0.7”* 


0.1 


2.1 


1. 7-2.6 


Over age 14 


-0.5”* 


0.1 


0.6 


0. 5-0.7 


One standard deviation above the mean on grade 8 math assessment 


04*** 


0.1 


1.4 


1.2-1 .6 


One standard deviation above the mean on grade 8 reading assessment 


04*** 


0.1 


1.5 


1.3-1 .7 


Cohort 


0.2 


0.2 


1.2 


0.8-1. 8 



’’•Significant at the 0.001 level. 

Source: Authors' analysis of student records provided by District A. 
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TABLE E2 

Regression analysis results for District B 



Predictor 


Estimated 

coefficient 


Standard 

error 


Odds 

ratio 


Odds ratio: 
95 percent 
confidence 
interval 


On-track 


1 7 *** 


0.1 


5.5 


4.4-7.0 


Female 


0.5*** 


0.1 


1.7 


1 . 4-2.1 


Asian 


0.3 


0.3 


1.5 


0.8-2.4 


Black or Hispanic 


-0.4*** 


- 0.1 


0.6 


0 . 5-0.8 


Identified as receiving free or reduced-price lunch 


- 0 . 6 *** 


0.1 


0.6 


0.4-0.7 


Over age 14 


-0.7*** 


0.1 


0.5 


0.4-0.6 


One standard deviation above the mean on grade 8 math assessment 


0 . 6 *** 


0.1 


1.7 


1 . 5-2.1 


One standard deviation above the mean on grade 8 reading assessment 


0.1 


0.1 


1.1 


0.9-1 .3 


Cohort 


0.1 


0.1 


1.1 


0.9-1 .3 



***Significant at the 0.001 level. 

Source: Authors' analysis of student records provided by District B. 
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NOTES 

1 . 



2 . 



3. 



Stillwell, Sable, and Plotts (2011) applied 
the averaged freshman graduation rate, a 
method used by the National Center for 
Education Statistics to estimate the percent- 
age of public high school freshmen who 
graduate with a regular diploma four years 
after starting grade 9 (Seastrom et al. 2006a, 
2006b). Under this method, the rate for 
2008/09 (the freshman class in the fall of 
2005) equals the total number of diploma 
recipients in 2008/09 divided by the average 
enrollment of grade 8 students in 2004/05, 
grade 9 students in 2005/06, and grade 10 
students in 2006/07. 

High-poverty schools are defined as schools 
with 76-100 percent of their enrollment 
eligible for free or reduced-price lunch. Low- 
poverty schools are those with 0-25 percent 
of eligible students. The graduation rate data 
were drawn from the 2007/08 Public School 
Data File of the Schools and Staffing Survey. 

Twenty-four states reported having student- 
level early warning indicator systems or 
readiness reports in the 2009/10 survey 
conducted by Data Quality Campaign (2010). 
Districts also are increasingly mining and 
analyzing their data sets to identify specific 
indicators that provide early signals that 
students are falling off track to graduation. 
Some well known examples are Chicago 
Public Schools (Allensworth and Easton 
2005, 2007), Baltimore City Public Schools 
(Mac Iver and Mac Iver 2010), Boston Public 
Schools (Pinkus 2008), the School District 
of Philadelphia (Neild, Balfanz, and Herzog 
2007), and Prince George’s County Public 
Schools in Maryland (Prince George’s County 
Public Schools 2010). 

The four-year graduation rate is of particular 
interest because it is currently the only gradu- 
ation rate monitored for federal accountability 
purposes. 



5. The third district interested in this study was 
unable to provide the complete data necessary 
for analysis. 

6. The findings in this section do not include grad- 
uation rates for students whose on-track status 
could not be determined at the end of grade 9 
because their course records were incomplete. 
For the District A sample, these include 72 stu- 
dents in the 2005/06 cohort (2.7 percent of the 
sample), of which 11 (15.2 percent) graduated on 
time, and 42 students in the 2006/07 cohort (1.7 
percent of the sample), of which 10 (23.8 per- 
cent) graduated on time. These graduation rates 
should be interpreted with caution because the 
study team was unable to determine why some 
of these students had incomplete course records 
or whether these students were on track or off 
track at the end of their first year of high school. 

7. The findings in this section do not include 
graduation rates for students whose on-track 
status could not be determined at the end of 
grade 9 because their course records were 
incomplete. For the District B sample, these 
include 81 students in the 2005/06 cohort (3.3 
percent of the sample), of which 33 (40.7 per- 
cent) graduated on time, and 59 students in 
the 2006/07 cohort (2.4 percent of the sample), 
of which 23 (38.9 percent) graduated on time. 
These graduation rates should be interpreted 
with caution because the study team was un- 
able to determine why some of these students 
had incomplete course records or whether 
these students were on track or off track at the 
end of their first year of high school. 

8. The results for the first two research questions 
show higher on-track rates and on-time grad- 
uation rates for District B than for District A 
(see appendix C). It is not within the scope of 
this study to compare these two districts or to 
explain on-track rates or on-time graduation 
rates for either district. 

9. To further examine the predictive strength 
of the on-track indicator, a model similar to 



30 



USING THE FRESHMAN ON-TRACK INDICATOR TO PREDICT GRADUATION IN TWO URBAN DISTRICTS 



that described in appendix E was run for each 
district, but with on-time graduation pre- 
dicted only by on-track status. The difference 
between the odds of on-time graduation in 
models that included baseline student back- 
ground characteristics (the primary model 
described in this section and appendix E) and 
in models that excluded baseline character- 
istics (reported in this footnote) suggest that 
student background characteristics, prior test 
scores, and cohort accounted for part of the 
relationship between on-track status and on- 
time graduation. For District A, when only 
on-track status was used as a predictor, the 
odds of on-time graduation for students who 
were on track at the end of their freshman 
year was estimated to be 7.84 times that of 
students who were off track at the end of their 
freshman year (with a 95 percent confidence 
interval of 6.62-9.30). This is larger than the 
odds of on-time graduation found in the pri- 
mary model reported in this section, which 
estimated an odds ratio of 6.57. For District 
B, when on-track status was used as the only 
predictor, the odds of on-time graduation 
for students who were on track at the end of 
their freshman year was estimated to be 9.25 
times that of students who were off track at 
the end of their freshman year (with a 95 
percent confidence interval of 7.42-11.54). 
This too is larger than the odds of on-time 
graduation found in the primary model 
reported in this section, which estimated an 
odds ratio of 5.54. The finding that the odds 
ratios estimated using only on-track status 
are larger than the odds ratios estimated with 
the student background characteristics is the 
expected result if student background char- 
acteristics are themselves related to on-time 
graduation. 

10. When students with missing test scores were 
included in the model using imputed values 
for the missing scores, estimated coefficients 



varied by less than 2 percent and retained 
their statistical significance; see tables C3 and 
C4 in appendix C. 

11. Data needed to analyze five-year graduation 
rates in Districts A and B were not available at 
the time of data collection. 

12. ffierarchical linear models delete observations 
with missing data on any level 1 variable. The 
results of a model that included all students 
and that estimated missing grade 8 test scores 
through multiple imputation procedures 
using student demographic data were com- 
pared with the results of the model for the 
analytic sample from the study (excluding 
students with missing test scores) and the full 
sample (including students with missing test 
scores by using imputed values). All estimated 
coefficients differed by less than 2 percent. It 
is possible that imputation using academic 
variables, such as grade 8 grade point average 
or grade 7 tests scores, would yield different 
results. However, data for these academic vari- 
ables were not available in the study dataset. 

13. Hierarchical models are most appropriate in 
cases where the level 1 intercept varies signifi- 
cantly across level 2 units. This was the case 
for District A, where between-school variation 
was significant at p< 0.05 but was not the 
case for District B. In this study, a hierarchi- 
cal model was used to take into account the 
clustering of students within schools and to 
obtain more conservative standard errors, 
reducing the risk of Type I error. 

14. The regression results indicate that once on- 
track status, other student background char- 
acteristics, and grade 8 assessment test scores 
were taken into account, higher graduation 
rates were predicted for non-White students 
than for White students and for students with 
an IEP than for students without one. 
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